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The expert system computer simulation detailed in 
this report is designed to investigate patterns of valuing in 
decision-making about dossiers of performance data used in teacher 
promotion evaluations. The knowledge base for the simulation consists 
of 111 questions organized into 117 rules. The questions and linked 
advice were derived from questionnaires completed by human judges of 
teacher dossiers, and the knowledge base is organized into sub-bases 
for five phases of dossier analysis: (1) preliminary screening; (2) 
analysis of nine individual data sources; (3) determination of data 
safety; (4) promotion decisions; and (5) comparison of two dossiers. 
This report lists the questions and rules for each of the nine 
individual data sources: (1) student achievement; (2) administrator 
reports; (3) other or unique data; (4) parent surveys; (5) peer 
reviews of materials; (6) professional activities; (7) student 
surveys; (8) systematic observations; and (9) teacher tests. It is 
noted that two judgments may be derived from the application of this 
system — to promote or to deny promotion, and the ranking of dossiers. 
The expert shell system used in this simulation design was MacSMARTS, 
and it was used on an Apple Macintosh microcomputer. (EW) 
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EXPERT SYSTEM KNOWLEDGE BASE FOR A COMPUTER SIMULATION 

OF 

JUDGMENTS ON DOSSIERS OF SCHOOLTEACHER PERFORMANCE 



Teacher evaluation is an informed/ rational, subjective human 
activity. The purpose for design of this simulation was to 
investigate patterns of valuing in human expert decision-making 
about teacher dossiers of performance data. The expert system uses 
many of the same decision strategies as human judges, but also 
employs some not used by hiimans. Information about these strategies 
may be useful for training of judges for dossier evaluation. 

The expert system shell used in this simulation design was 
MacSMARTS from Cognition Technology, 55 Wheeler Street, Cambridge, 
MA 02138. MacSMARTS uses a Prolog inference engine. MacSMARTS 
requires 134K of space. The computer was an Apple Macintosh Plus 
with 800K internal and external disk drives. 

The knowledge base for the simulation design consists of 111 
questions organized into 117 rules. The questions and linked advice 
were derived from questionnaires completed by human judges of 
teacher dossiers. These judges were from two school district 
promotion panels, students in a university course on teacher 
evaluation, and several university education faculty. The knowledge 
base requires 382K of space. The knowledge base is organized into 
sub-bases for five phases of dossier analysis: preliminary 
screening, analysis of nine individual data sources, determination 
of data safety, promotion decision, and comparison of two dossiers. 
Two judgments may be derived from this application: 1) Promote or 
Deny and 2) ranking of dossiers. The knowedge base wall be refined 
in later versions. 



The sub-bases include: 







Rules 


Questions 


Size 


I. 


Initial screening 


5 


4 


17K 


II. 


Student achievement 


7 


8 


23K 


II. 


Administrator report 


6 


5 


20K 


II. 


Other, unique 


5 


13 


17K 


II. 


Parent survey 


7 


8 


23K 


II. 


Peer review of materials 


10 


10 


33K 


II. 


Professional activity 


12 


9 


39K 


II. 


Student survey 


6 


7 


20K 


II. 


Systematic observation 


9 


11 


29K 


II. 


Teacher tests 


10 


9 


33K 


III. 


Data safe? 


12 


9 


39K 


IV. 


Promote or Deny 


9 


6 


29K 


V. 


Compare 


19 


12 


60K 
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RULES: FOR I, INITIAL SCREENING 

1 Rssess aach line with appropriate knowledge base, then use PROMOTION DECISION knowledge base. 
IF VES: Rt least 3 years experience, «ith 1 in district? 

IF VES: Dossier no more than 17 pages? 

IF VES: Evidence appears in required, coiriparable formats <forms, procedures)? 
IF VES: Evidence appears to be serious, good-faith? 

2 DENV. Uait until Minimum tenure met. 

IF HO: fit least 3 years experience, with 1 in district? 

3 DENV. Wait until dossier is shortened to maximum of 17 pages. 
IF HO: Dossier nc more than 17 pages? 

4 Refer dossier to ProiRotion Panel for decision. 

IF NO: Evidence appears in required, coitporoble formats (forms, procedures)? 

5 DENV. Ho hoaxes, please. 

IF NO: Evidence appears to be serious, c,oo<)-fai th? 



RULES: FOR II. ACHIEVEMENT 

1 PftSS: EXCEPTIOt«L 

IF VES: Goals & measure selected & validated as per district system? 

IF VES: Do pre- and post- measures sho« pupi I gain adjusted for prior achievement? 

IF VES: fire descriptions of educational significance present? 

IF VES: fire gain data significantly better than competitors? 

IF VES: Have multi-year data been presented (rtiich show an exceptional pattern of achievement? 

2 PflSS 

IF VES: fire significant gain data presented? 

IF VES: OoaJs & measures selected & validated as per district system? 

IF VES: fire pre- and post- measures used? 

IF VES: Do goin data ioke into account prior achievement? 

IF VES: fire descriptions of educational signficance present? 

3 PflSS: BflRaV 

IF VES: fire significant gain data presented? 

IF VES: Goals & meosures selected & validated as per district system? 

IF VES: Are pre- and post- measures used? 

IF NO: Do gain data toke into account prior ochievement? 

IF VES: fire descriptions of educational signficance present? 

4 PftSS: BflRELV 

IF VES: fire significant gain data presented? 

IF VES: Goals & measures selected & validated as per district system? 

IF NO: fire pre- and post- measures used? 

IF VES: Do gain data take into account prior achievement? 

IF VES: Rre descriptions of educational significance pres<5nt? 

5 FAIL: aOSE 

IF VES: fire significcr)t gain data presented? 

6 FAIL 

IF NO: Goals & measures selected & validated as per district system? 

7 FAIL 

IF HO: Are significant gain data presented? 
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PftSS: BflRELV 

IF VES: "...with exemplary practice"? 
IF VES: fldiBtnistrator identified? 
IF HO: Host recent year Included? 

PRSS 

IF VES: "...with exemplary practice"? 
IF VES: fidalnlstrator identified? 
IF VES: Most recent year Included? 

PftSS 

IF VES: "contributing, itfell functioning"? 
IF VES: fldmlnlstrator identified? 
IF VES: Host recent year included? 

FfilL: CLOSE 

IF VES: "contributing, well functioning" Ctfl "...with exemplary practice"? 
IF NO: fldainistrator Identified? 

FftIL: aOSE 

IF VES: "contributing, wfil I functioning" CR "...with exemplary practice"? 
IF MO: Most recent year included? 

FftIL 

IF NO: "contributing, oel I functioning" OR "...with exemplary practice"? 



RULES: FOR II. OTHER ^ UNIQUE 

1 FftIL. Combine this evidence with another line. 

IF HO: Is the evidence distinct from other lines, i.e., dearly not fit in? 

2 FftIL. Evidence is not equivalent of other lines. 

|p !2: ^.Jlf^'''^^'' Pos'^'o"' e.g., «edia, counselor, pull-out teacher, nurse? 

r 22" S ^'fnif leant teacher leadership function, »ith effects beyond the classroom? 

P m- TJ" 'f?'^*»°,^e audience other than students, parents, peers, administrators? 

F NO. Educational function not usually associated with classroom teaching^ 

IF NO: ft creative contribution to the educational system? 

^ i?'^N0-^n!f?2 Tl^^^ ^f'^^'^i effect on a legitimate audience. 

IF NO. Did the effort educationally benefit students? 

IF HO: Did the rffort benefit teachers in the system? 

IF NO: Did the effort benefit parents in their support of the system? 

F nS" I!!! ^VJ^\ ^'"^ significant additional resources and/or recognition to the system? 

l^ NO: Did the effort increase educational opportunities for students? 

4 FftIL 

IF HO: Here the data gathered and reported with credibility, using safe procedures? 

5 pass 

pi?^ ^"""^ benefitted from the effort: students, teachers, parents, administrators, 



RULES: FOR II. PARENT SURVEY 



1 PftSS: EXCEPTlONflL 

IF VES: Global rating 1 SO above the mean OR 4.9? 

IF VES: If jta^ior or senior high, mere at least 3 classes per year used? 

IF VES: Tnree year^ of data included? 

IF VES: FoTfi, data gathering, scoring and reporting safe? 

IF VES: Return rate greater than 80» fifC information supplied greater than requested? 

2 PflSS 

IF VES: Global rating obove 1.5 SO below the »ean? 

IF VES: If junior or senior high, ©ere at least 3 classes per year used? 

IF VES: FoTft^ data gathering, scoring and reporting safe? 

IF VES: Return rate greater than 60* RNO infomiation supplied greats than requested? 

IF VES: Rll items above 3.3? 

3 PflSS: Bflra^V 

IF VES: Global rating above 1.5 SO belo«^ the wean? 

IF VES: If junior or senior high, apere at least 3.classes per yecM^ used? 
IF VES: Form, data gathering, scoring crrf reporting safe? 

IF VES: Return rate greater than 6011 AND information st^^lied greater than requested? 
IF NO: Rll items above 3.3? 

4 FAIL: aOSE 

IF VES: Global rating <d>ove 1.5 SO below the mean? 

IF MO: If jtoiior or senior l^igh, mere at least 3 classes per year used? 
IF VES: Form, data gathering, scoring and reporting safe? 

IF VES: Return rate greater than &0f fltC information supplied greater than requested? 

5 FAIL 

IF NO: Form, data gathering, scoring ond reporting safe? 

6 FRIL 

IF NO: Return rate greater than 60* AND information supplied greater than requested? 

7 FAIL 

IF NO: Global rating above 1.5 SO below the mean? 
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RULES: ^OR II, P3ER REVI2W OF MATERIALS 

1 PfiSS: EXCEPT !(»iftL. 

IF VES: Did peers recommend promotion fViD report evidence of exemplary practice? 

IF VES: Did the »Gterials contain curricuium, instruction, and student wotk ^ outcomes? 

IF YES: Did the Mterials cover work done over more than one year? 

IF VES: Review credibly organized, si^Dervised, «ith 3 knowledgoble, social/political detached peers? 
IF VES: Was the review conducted within the post five. years? 

2 pass 

IF VES: Did peers i-^ecoamend promotion? 

IF VES: Did the pwlerials contain curriculua, instruction, and student «K>rk & outcomes? 
IF VES: 3 peers knowledgeable of school & subject /grade, but politically/socially detached? 
IF VES: Has the review credibly organized ano supervised? 
IF \tS: Has the review conducted within the post five years? 

3 PfiSS: BflRELV 

IF VES: Did peers recoaaend proiiotion? 

IF MO: Did the natertais contain curriculum, instruction, and student work & outcomes? 
IF VES: 3 peers knc«iedgeable of school & subject/grade, but politically/socially detached? 
IF VES: UoB the review credibly organized and supervised? 
IF VES: Uas the review conducted within the past five years? 

4 FfilL: CLOSE 

IF VES: Did peers recooMend promotion? 

IF VES: Did the Materials contain curriculum, instruction, and student work & outcomes? 
IF HO: 3 peers knowledgeable of school fe subject/grode, but politically/socially detached? 
IF VES: Uas the review credibly organized and supervised? 
IF VES: Uas the review conducted within the post five yecn^? 

5 FAIL: aOSE 

IF VES: Did peers recommend promotion? 

IF VES: Did the materials contain curriculum, instruction, and student mrk ^ outcomes? 

IF VES: 3 peers knowledgeable of school & subject /grade, but politically/socially detached? 

IF HO: Was the review credibly organized and supervised? 

IF VES: Uas the review conducted within the past five yeors? 

6 PfiSS: BflRELV. Hinimum conditions met. 
IF VES: Did peers recommend promotion? 

IF VES: Did the materials contain curriculum, instruction, and '>tudent work outcomes? 
IF VES: 3 peers knowiedge<ri)le of school & subject /grade, but politically/socially detached? 
IF VES: Uas the review credibly organized and supervised? 
IF VES: Uas the review conducted between five and seven years ago? 

? FfilL 

IF NO: Did peers recommend promotion? 

8 FfilL 

IF NO: 3 peers knowledgeable of school & subject/grode, but politically/socially detached? 

9 FAIL 

IF NO: Uas the review credibly organized and supervised? 

10 FfilL 

IF VES: Did the review occur more than seven years ago? 
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RULES: FOR II ♦ PROFESSIONAL ACTIVITY 



1 PASS: EXCEPT I OtiflL 



IF VES: 
IF VES: 
IF VES: 
IF VES: 
IF S'ES: 

PASS 
IF VES: 
IF HO: 
IF VES: 
IF VES: 
IF VES: 

PflSS 
IF VES: 
IF VES: 
IF NO: 
IF VES: 
IF VES: 

PflSS 
IF VES 
IF VES 
IF VES 
IF VES 
IF VES 



Active in 5 of : inservice Jeadership, prof, groups, communi ty groups, col leagues, C&t (J) 
2,> areas: involved, consistent, cont'-ibuting, high quality, remarkable? 
Is substantial inservice education included? 

Does evidence show a pattern of consistency and long-term involvement? 
Is evidence well gathered, documented and credible? 

Active in 5 of: insen,- ice, leadership, prof, groups, comrouni ty groups, col leagues, C& I (j) 
2,> areas: involved, consistent, contributing, high quality, remarkable? 
Is substantial inservice education included? 

Does evidence show a pattern of consistency and long-term involvement? 
Is evidence veil gathered, documented caid credible? 

Active in 5 of : inservice, leadership, prof, groups, communi ty groups, col leagues, C& I (T) 
2,> areas: involved, consistent, contributing, high quality, remarkable? 
Is substantial inservice education included? 

Does evidence shotu a pattern of consistency and long-tern: involvement? 
Is evidence well gathered, documented and .o-edible? 

Active in 3-5 of : inservice, leadership, prof, groups, community groups, col leagues, C& I (j) 
1,> area: involved, consistent, contributing, high qtial i ty, remarkable? 
Is substantial inservire education included? 

Does evidence show a pattern of consistency and long-term involvement? 
Is evidence well gathered, documentad and credible? 



5 PflSS: BffiELV 



IF VES 
IF NO 
IF VES 
IF VES 
IF VES 



Active in 3-5 of: inservice, leadership, prof, groups, community groups, col leagues, C8i I (T) 
1,> area: involved, consistent, contributing, high quality, remarkable? 
Is substantial inservice educatiori included? 

Does evidence show a pattern of consistency and long-term involvement? 
Is evidence uiel I gathered, docut^ented and credible? 



6 PASS: 8ARELV 



IF VES 
IF VES 
IF NO 
IF VES 
IF VES 



Active m 3-5 of : inservice, leadership, prof, groups, communi ty groups, col leagues, C& I (j 
1,> area: involved, consistent, contributing, high quality, remarkable? 
Is substantial inservice educaxion included? 

Does evidence show a pattern of consistency arid long-term involvement? 
Is evidence well gathered, documented and credible? 



7 PASS: BARELV 



IF VES 
IF VES 
IF VES 
IF VES 
IF VES 



Data in 2 of: inservice, leadership, prof, groups, community groups, col leagues, C«i I 
Is activity in at least 1 area: involved, consistent, contributing, high quality? 
Is inservice education included? 

Does evidence shora a pattern of consistency and long-term involvement? 
Is evidence well gathered, documented and credible? 



0 



8 FAIL: aOSE 



in 2 of: inservice, leadership, prof, groups, community groups, col leagues, C& I 
rea: involved, consistent, contributing, high quality, remarkable? 



IF VES: Data 
IF VES: 1,> area: 

IF NO: Is inservice education included? 
IF VES: Oo«fc evidence show a pattern of consistency and long-term involvement? 
IF VES; Is evidence well gathered, docuniented and creoible? 



(continued, next page) 
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9 FRIL: aOSE 

IF VES: Data in 2 of: inserv ice .leadership, prof, groups, community groups,colleagues,C&l 0 

IF VES: t^> area: involved, consistent, contributing, high quality, reffiarkobie? 

IF VES: Is inservice education included? 

IF NO: Does evidence show a pattern of consistency and loTtg-term involvement? 

IF VES: Is evidence well gathered, docutfiented and credible? 

10 FAIL 

IF NO: Data in 2,> of: inservice, leadership, prof, groups, coronjunity groups, co II eGgues,C8cI (f) 

11 FAIL 

Ir NO: Is activity in at least 1 area: involved, consistent, contributing, high quality? 

12 FAIL 

IF NO: Is evidence well gathered, documented and credible? 



^ •development r advanced degrees or certificates? 



RULES: FOR II. STUDENT SURVEY 

1 PflSS: EXCEPTIONflL 

IF VES: Global rating 1 SO above the neon (Xi 4.9 (whichever is lou;er>? 

!F VES: If junior or senior hi^, mere at least 3 classes per year used'^ 

IF VES: Were three years of data included? 

IF VES: Here the form, data gathering, scoring and reportinq safe'^ 

IF VES: fill items above 4.0? ^ ^ • 

2 PflSS 

IF VES: Global rating above 1.5 below the »ean? 

IF VES: If junior or senior high, were at least 3 classes per year used? 

IF VES: Here the for©, data gathering, scoring and reportinq safe'^ 

IF VES: AM items above 3.3? 

3 PftSS: BflRELV 

IF VES: Global rating obove 1.5 SO below the mean? 

IF VES; If junior or s«iior high, were at least 3 classes per year used? 
IF VES: Were the form, data gathering, scoring and reportinq safe'^ 
IF HO: Rii items above 3.3? 

4 FAIL: aOSE 

IF VES: Global rating above 1.5 SO below the mean? 

IF NO: If junior or senior high, were at least 3 classes per year used? 
IF VES; Were the form, data gathering, scoring and reporting safe? 

5 FAIL 

IF HO: Were the form, data gathering, scoring and reporting safe? 

6 FAIL 

IF HO: Global rating above 1.5 SO below the mean? 



FOR II. SYSTEMATIC OBSERVATION 
1 FAIL 

IF NO: Observer trained, Ronitored, social ly-pol i tical ly neutral? ^ 




2 


FftIL 

IF NO: More than 4, unannounced^ representative visits? 




3 


IF rW: Analyst identified, expert, S reliable? 




4 


FRiL 

IF NO: Recording ^ analysis systems defensible? 




5 


FAIL 

IF NO: Data & reporting safe? 




6 


PASS- EXEMPLflRV 

IF VES: USE OF TIIC caxl OPPORTUHITV TO LEfiRN "exerapiory"? 

IF VES: Rll categories "recownend" or "exemplary"? 

IF VES: fit least 4 categories included? 

IF VES: Absence of aajor category explained? 
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PflSS 

IF VES: USE OF TIME and OPPORTUHITV TO LEflWi included? 

IF VtS: AM CateOQI^ipS "r^r^l^MhoryH** n*^ "a's^amnlrw^t^'O . 

IF VES: Hore Ukw 2 categories included? 
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PflSS: BflRELV 

IF VES: fill categories "recommend" or "exemplary"? 




9 


FfilL. Quality not showi. 

IF MO: fill categories "recomnend" or "exeaplcry"? 




RULES: FOR II. TEACHER TESTS 




1 


FfilL 

IF NO: Pre the test data safe? 




2 


FfilL 

IF VES: Is the score below the 50«ile? 




3 


FfilL 

IF NO: Is the test appropriate for the teacher (grade level, subject area)? 




4 


FAIL 

IF NO: Is the test one of the following <or equivalent) : HYE, 6RE, SCAT, CBEST? 




5 


FAIL 

IF HO: Mas \t\e test taken within 9 years? 




6 


PflSS: EXCEPTIONAL 

IF VES: Is the score in the 90-99«i le range? 
IF VES: Has the test taken within 5 years? 




7 


PflSS 

IF VES: Was the test taken within 9 years? 
IF VES: Is the score in the 90-99«i ie range? 




8 


PflSS 

IF VES: Is the score in the 60-89«i le rcaige? 
IF VES: Uas the test taken within 5 years? 




9 


FAIL: aOSE 

IF VES: Is the score in the 50-59Xile range? 




to 
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FfilL: CLOSE 

IF VES: Is the score ;n the &0-89fiie range? 

IF NO: Mas the test taken within 5 years? jQ 

IF VES: Uos the test token within 9 years? 
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RULES: FOR III. ARE DATA SAFE? 

1 DRTfl ARE SflFE 

IF VES: Iters fcras, surv&^, S reports stcsttdord, district-ailopted? 

IF VES: Here data collected, stored, reported by on identified, credible third party? 

IF VES: Mere pca^ticlpants identified, available for confirmation contoct? 

IF HO: fire there any doubts about the honesty and fairness of the data? 

2 DflTfl ARE SftFE 

IF !10: Here foms, surveys, & reports standord, district-adopted? 

IF VES: Mere dola collected, stored, reported by on identified, credible third party? 

IF VES: Mere participants identified, available for confirmation contact? 

IF MO: fire there any doubts obout the honesty and fairness of the data? 

IF VES: Alternate forms give good validity & reliability data; RHD, useage explanation? 

3 DATA ARE S«^E 

IF VES: Mere forms, surveys, & reports standard, district-adopted? 

IF ^B: M'ire data collected, stored, reported by an identified, credible third party? 

IF VES: Mere porticiponts tdentif'sed, ovailable for confirmation contact? 

IF NO: Are there any doubts about the honesty and fairness of the data? 

IF HO: Do data require collection, storage, & report by identified, credible third party? 

4 DATA ARE SAFE 

IF VES: Mere forms, surveys, & reports standard, district-adopted? 

IF VES: Mere dota collected, stored, reported by an identified, credible third party? 

IF HO: Mere participants identified, available for confirmation contact? 

IF HO: Are there ony doubts about the honesty and fairness of the dota? 

IF HO: Do these data require that participants be identified, available for confirmation contact? 

5 DATA ARE HOT SAFE 

IF VES: Are there any doubts about the honesty and fairness of the data? 

5 Probably safe, but request an explanation to make sure. 

IF HO: Mere forms, surveys, 6 reports standard, district-adopted? 

IF VES: Do alternate forms, surv&^, & reports give good validity & reliability data? 

IF HO: Is there an explanation for alternate useage? 

7 DATA HOT SAFE 

IF HO: Mere forms, surveys, & reports stondard, distr ict-odopted? 

IF HO: Do alternate forms, surveys, & reports give good vol idi ty & rel iobi I ity data? 

8 DATA HOT SflFE 

IF HO: Mere data collected, stored, reported by an identified, credible third party? 

IF VES: Do data require collection, storage, S report by identified, credible third party? 

9 DATA HOT SAFE 

IF HO: Mere participants identifijd, available for confirmation contact? 

IF VES: Do these data require that participants be identified, aval loble for confirmation contact? 

10 DATA HOT SAFE 

IF HO: Mere forms, siMTK^ei^s, & reports standard, distr ict-odopted? 

1 1 DATA HOT SflFE 

IF HO: Mere data collected, stored, reported by an identified, credible third party? 

12 DATA NOT 

IF HO: participants identified, available for confirmation contact? 
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FMiSr .FOR IV,, PROMOTE OR DENY 

1 PROMOTE 

!F VES- Does the dossier have at least 4 lines of evidence which PftSS <at any IsveD? 

2 PROMOTE 

IF NO: Does the dossier hove at least 4 lines of evidence which PRSS <at any level)? 
IF VES: fire there 3 lines which PfiSS (not BfiRELV) cr>d at least 1 which is FAIL: aOSE? 
IF VES: Is there at least 1 PRSS: EXCEPTIOMRL? 

3 PROMOTE 

!F NO: Does the dossier have at least 4 lines of evidence which PASS (at any level)? 
IF VES: Are there 3 lines which Pfl?«: EXCEPT I OTiftt.? 

4 PROMOTE 

IF NO: Does tte dossier hove at least 4 lines of evidence which PRSS (at orty level)? 
IF VES: fire thore 3 lines which PflSS (not BRRELV)? 

IF VES: Is there at least one PflSS tine which is truly outstanding and unusual? 

IF VES: Does outstanding & unusual line impact exter^l beyond this teot^ier's classroow? 

5 DENV 

IF NO: Does the dossier have at least 4 lines of evidence which PRSS (at any level)? 
IF NO: fire there 3 lines which PflSS (not BRRELV) and at least 1 which is FAIL: aOSE? 

6 DENV 

IF NO: Does the dossier have at least 4 lines of evidence which PASS (at any level)? 

IF VES: are there 3 lines which PftSS (not BARELV) ond at least 1 which is FAIL: CLOSE? 

IF NO: Is there at leost 1 PflSS: EXCEPTIONAL? 

7 DENV 

IF NO: Does the dossier have at least 4 lines of evidence which PASS (at any level)? 

IF NO: Are there 3 lines which PfiSS (not BRRELV)? 

IF VES: Is there at least one PRSS line which is truly outstanding and unusual? 

IF VES: Does outstta^ing & tauisual line iitpact extend beyond this teacher's classroom? 

8 DENV 

IF NO: Does the dossier have at least 4 lines of evidence which PRSS (at any level)? 

IF VES: Rre there 3 I ines which PRSS (not BRRELV)? 

IF HO: Is there at least one PRSS line which is truly outstanding and unusual? 

9 DENV 

IF NO: Does the dossier have at least 4 lines of evidence which PRSS (at any level)? 

IF TO: fire there 3 lines which PflSS (not BRRELV)? 

IF TO: Is there at least one PRSS line which is truly outstanding and unusual? 

IF TO: Does outstanding & unusual line impoct extend beyond this teacher's classroow? 




RUl.ES: FOR V. COMPARTS 

1 n>B 

IF NO: Did both receiue the same PROMOTE /DENV? 
IF VES: Did fl receive PROMOTE? 

2 fl>B 

IF VES: Do both hove somr lines of evidence? 

IF VES: Do they differ or more ratings (e.g., PflSS, FAIL: CLOSE)? 

IF VES: Does fl rate ' on at least 1 I ine, and lower on none? 

3 fl>B 

IF VES: Do both have same I ines of evidence? 

IF VES: Do they differ on 1 or more ratings (e.g., PASS, FfllL: CLOSE)? 

IF VES: Is fl superior on ©ore I ines than B? 

4 fl>B 

IF VES: Do both have some I ines of evidence? 

IF VES: Do they differ on 1 or more ratings (e.g., PflSS, FfllL: CLOSE)? 

IF MO: Is fl superior on Rore I ines thun B? 

IF VES: Is fl superior on STUDENT flCHIEUETOfr, PEER REUIEU, STUDENT SURUEV »hen they appear? 

5 fl>B 

IF VES: Do both have same I ines of evidence? 

IF VES: Do they differ on 1 or more ratings (e.g., PflSS, FfllL: CLOSE)? 

IF VES: fire the --esults mixed, i.e., each is higher on some? 

IF HO: Same points: achievement <9), peer & pupil <8>, observation & tests 8 activity (7), parents 

IF VES: Does fl have more points? 

6 fl>B 

IF NO: Do both hove same I ines of evidence? 

IF VES: Do they differ on 1 or more ratings (e.g., PflSS, FfllL: CLOSE)? 

IF VES: fire the results mixed. I.e., each is higher on some? 

IF VES: Same points: achievement <9), peer & pupil <8), observation & tests & activity (7), parents 

IF VES: Does fl have more points? ^ 

7 fl.B 

IF HO: Do both have same * of lines of evidence? 

IF HO: Do they differ on 1 or more ratings (e.g., PflSS, FfllL: CLOSE)? 

IF HO: Same points: achievement (9), peer & pupil (8), observation & tests & activity (7), parents 
IF VES: Does fl have more points? q 



8 B>ft 
IF 
IF 



NO: Did both receive the some PRW10TE/DENV? 
HO: Did ft receive PROMOTE? 



9 B>fl 

IF VES: Do both have same lines of evidence? 

IF VES: Do they differ on 1 or more ratings (e.g., PflSS, FfllL: CLOSE)? 

IF HO: Does fl rate higher on at least 1 line, and !o»er on none? 

10 B>fl 

IF VES: Do both have same I ines of evidence? 

IF VES: Do they differ on 1 or more ratings (e.g., PflSS, FfllL: CLOSE)? 

IF HO: Is fl superior on more lines than B? 

11 B>fl 

IF VES: Do both have scse I ines of evidence? 

IF VES: Do they differ on 1 or more ratings (e.g., PflSS, FfllL: CLCKE)? 

IF HO: Is fl superior on more lines than B? 

IF HO: Is fl superior on STUDENT flCHIEUEHENT, PEER REUIEU, STUDENT SURUEV when they appear^? 
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12 

12 B>R 

IF VES: Do both have same lines of evidence? 

IF VES: Do they differ on 1 or more ratings <e.g., PRSS, FfilL: CLOSE)? 
IF VES: fire the results mixed, i.e., each is higher on some? 

IF HO: Same points: achievement <9), peer & pupil <8), observation & tests & activity <7), parents 
IF NO: Does fi have more points? ^ 

13 B>R 

IF HO: Lo both have some lines of evidence? 

IF VES: Do they differ on 1 or more ratings <e.g., PflSS, FfilL: CLOSE)? 

IF VES: fire the results mixed, i.e., each is higher on soffie? 

IF VES: Some points: achievement <9), peer & pupil <8), observation & tests & activity <7), parents 

IF HO: Does fi hove more points? 

14 B>fl 

IF HO: Do both have same * of I ines of evidence? 

IF HO: Do they differ on 1 or more ratings <e.g., PftSS, FfilL: CLOSE)? 

IF HO: Some points: achievement <9), peer & pupil <8), observation 8 tests & activity <7), parents 

IF HO: Does fi hove more points? Q 

15 TIE 

IF VES: Did both receive the same PROHOTE/DEHV? 

16 TIE 

IF VES: Do both have same lines of evidence? 

IF HO: Do they differ on 1 or more ratings <e.g., PflSS, FfilL: CLOSE)? 

IF HO: Rre the results mixed, i.e., each is higher on some? 

IF VES: Same points: achievement <9), peer & pupil <8), observation & tests & octivity <7), parents 

17 TIE 0 
IF VES: Do both have same » of lines of evidence? 

IF VES: Do they differ on 1 or more ratings <e,g., PftSS, FfilL: CLO^)? 

IF VES: Same points: achievement <9), peer & pc^il <8), observation & tests & activity <7), parents 

18 TIE & 
IF HO: Do both have same « of I ines of evidence? 

IF HO: Do they differ on 1 or more ratings <e.g., PftSS, FftIL: CLOSl/> 

IF VES: Same points: achievement <9), peer & pupil <8), observation & tests & activity <7), parents 

19 TIE (T^ 
IF HO: Do both have some lines of evidence? 

IF VES: Do they differ on 1 or more ratings <e.g., PftSS, FftIL: aOSE)? 
IF VES: ftre the results mixed, i.e., each Is higher on some? 

IF VES: Same points: achievement <9), peer & pupil <8), observation & tests %c octivity <7), parents Q 



••(6), administrator & Other (5)? 
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